Articles written by male economists are cited less than articles published by women in the same journals, a new study on gender and quality in economics finds. The authors also find that men’s citations rise when they co-author with women, and that women’s citations fall while they co-author with men, conditional on acceptance.
Women are under-represented in top economics journals. Although they make up 20–30 percent of academic economists, women are only 11 percent of all authors published in top-five journals since 1990, 12 percent since 2000, and 14 percent since 2010 (see Figure 1). Over that same period, there has been very little growth in the number of exclusively female-authored papers; almost no growth in the number of majority female-authored papers; and no meaningful change in the number of mixed-gendered papers with a senior female author.
The only tepid growth that has occurred is largely—if not entirely—due to an increase in the number of published articles by senior men co-authoring with a minority of junior women.((Data from Hengel and Moon (2019).))
Several recent papers investigate why. Hengel (2019) finds female-authored papers in top-four economics journals are held to higher writing standards in academic peer review. As a result, their manuscripts are subject to greater scrutiny, spend longer under review and women, in turn, respond by conforming to those standards. Using submissions data from a set of four semi-overlapping journals, Card et al. (2020) find that female-authored papers are higher quality conditional on referee recommendations.
In a new study, we join this research by investigating gender differences in quality conditional on acceptance to a top-five journal. We find that (i) articles authored by men are cited less than articles the same journals publish by women; (ii) men’s citations rise when they co-author with women; and (iii) women’s citations fall while they co-author with men. Under strong—but we believe reasonable—assumptions, these results imply top economics journals hold female-authored papers to higher standards, and, consequently, do not publish the highest quality research. They also suggest that authors of both sexes will be less willing to collaborate with women, all else equal.
The Female Advantage in Citations
Results from our principle analysis are displayed in Figure 2. Pink markers (first panel) represent the average female advantage in citations, conditional on controls.((To proxy for quality, we follow previous research and use the inverse hyperbolic sine (asinh) of citations as our dependent variable (Card and DellaVigna 2017; Card et al. 2020). To define article gender, we use its proportion of female authors (for a justification of this indicator, see Hengel 2019). Results and conclusions are similar if we use alternative transformations or raw citation counts and/or define article gender by various other means—e.g., solo-female-authored or exclusively female-authored. For more information on these choices and results using alternative specifications, see Hengel and Moon (2019).)) Column (1) adjusts for year of publication, journal, co-author count, and authors’ experience at the time an article was written—all factors found in previous research to influence citations, conditional on quality.((See Hengel and Moon (2019) for select citations and Tahamtan, Afshar, and Ahamdzadeh (2016) for an extensive review of this literature.)) Column (2) adds a control for author reputation at the time citations were collected; it accounts for the impact a stronger reputation later on can have on citations to an author’s earlier work, all else equal. Columns (3)–(5) restrict the data to articles published after 1990. Columns (4) and (5) control for sub-field. Columns (6)–(7) are estimated on the sample of articles published after 2000 and control for journal and year, only.
Every model in Figure 2 suggests female-authored papers are cited more than male-authored papers. Figure 3 reinforces this conclusion. It plots the distribution of citations for solo-authored papers; consistent with results in Figure 2, women’s citations are, on average, higher than men’s citations.
Everyone’s Citations Rise When They Collaborate with Women
Figure 4 analyses how citations change as the same economist co-authors with members of the opposite sex. Pink figures (first panel) represent the returns to male economists from co-authoring with women; blue figures represent the returns to female economists from co-authoring with men. As in Figure 2, models differ by controls and sample restrictions.
Everyone’s citations rise when they collaborate with women, conditional on acceptance to a top-ranked journal (Figure 4, first panel). Men’s citations increase 10–12 log points when they co-author with an equal share of women (pink triangle). Women’s citations fall -12 to -19 log points when they co-author with an equal share of men (blue square). Male and female comparisons are relative to papers they either solo author or co-author entirely with other men and women, respectively.
Estimates in Figure 4 are potentially biased by contributions from unobserved co-authors. For example, suppose, hypothetically, that Anna Schwartz, Milton Friedman, and Kenneth Arrow collaborated on one paper while Arrow collaborated with Larry Summers on another. If Friedman’s contribution means Arrow’s article with Schwartz and Friedman is cited more than his article with Summers, then the ratio of female authors on his papers will spuriously correlate with citations unless Friedman’s presence is accounted for.
To adjust for this, we additionally restrict our sample to senior male economists with at least two top-five papers co-authored with a single junior author of each sex. This creates a treatment group—i.e., senior male authors co-authoring with exactly one junior woman—that very closely resembles the untreated group—i.e., those very same seniors co-authoring with exactly one junior man.
Consistent with results in Figure 4, we find a senior male author’s work is more highly cited when it is co-authored with a woman: citations increase 60–70 log points when senior male economists co-author with junior women as opposed to junior men.
What Does This Mean?
Assuming citations are not biased in favor of women conditional on quality (as well as included controls), our results suggest female-authored papers published in top economics journals are higher quality than male-authored papers.((Prior research suggests this assumption holds: homophily skews citations toward male authors (Ferber 1986; Dion, Sumner, and Mitchell 2018); economists believe female-authored papers are cited less, holding quality constant (Card et al. 2020); men are also more likely than women to cite their own work (King et al. 2017).))
But why? There are two possible explanations: (i) female-authored papers are higher quality, conditional on submission; or (ii) they’re only higher quality, conditional on acceptance—i.e., women’s papers are held to higher standards in peer review.
If female-authored submissions are higher quality, then their acceptance rates should be higher as well, accounting for similar controls.((This is only necessarily true if the quality of female-authored submissions is normally—or at least uni-modally—distributed (see Hengel and Moon (2019) for a more formal treatment). Under certain conditions, higher quality female-authored papers conditional on acceptance could also be consistent with gender-neutral acceptance standards if the quality of women’s (or men’s) submissions follows a multi-modal distribution.)) Although our data cannot directly test this, recent evidence from a set of journals that partially overlaps with our own suggests they aren’t. Other studies of and beyond economics journals reach similar conclusions.
Figures 4 and 5 also rule out consistent same- or opposite-sex complementarities meaningfully contributing to gender differences in quality, conditional on acceptance. The impact of co-authoring with members of the opposite sex would be positive for men and women if they produce higher quality work by collaborating with one another; conversely, both effects should be negative if everybody works better with members of their own sex. Our evidence is not consistent with either story. Instead, we find that the impact of co-authoring with a man is negative for women while the impact of co-authoring with a woman is positive for men. This suggests that women disproportionately contribute to the quality of co-authored work—again, conditional on acceptance—and is consistent with women facing tougher standards in peer review.
Finally, our results indicate that female economists may not be able to correct for higher standards all on their own. Although the fall in women’s citations when they co-author with men suggests adding male co-authors can alleviate higher standards, men also experience a rise in citations when they co-author with women, suggesting co-authorship alone probably cannot eliminate them. And when combined with evidence of the “Matilda effect” in tenure decisions—Heather Sarsons shows that tenure committees discount women’s contributions to mixed-sex co-authored papers—co-authoring with men may have wider negative consequences, as well.
Policy Conclusions?
As Becker (1957) has shown, discrimination hurts its victims and its perpetrators. In the presence of complete and perfect markets, harm inflicted on the former is fully absorbed into higher costs for the latter. As a result, bias has no effect on equilibrium outcomes—e.g., women can simply publish their higher quality papers in currently lower-tiered (but non-discriminatory) journals, confident that their actions would lead to an immediate adjustment in journal rankings.
When competition isn’t perfect, however, discrimination interacts with one or more market frictions to prevent those who discriminate from fully internalizing its costs. Consequently, its victims will have to partially bear them. For example, imperfect information about journal rankings may mean tenure and promotion committees’ expectations are slow to adjust to the lower quality of journals that reject too many women. As a result, women (and the men they co-author with) are tenured and promoted at lower rates than they otherwise would be if markets were complete. To the extent that grant committees similarly rely on applicants’ past publication histories to choose between projects, women will also have a harder time funding future work.
Moreover, discrimination undoubtedly distorts authors’ decisions in ways that can further misallocate available resources. Indeed, our own evidence implies male and female economists are better off collaborating with men—all else equal—suggesting authors of both sexes may forgo higher quality co-authoring opportunities with women in order to partner with men.
In economics, we generally approach incomplete markets by proposing policies that try to fix individual imperfections. When the space of market failures is large and each is impossible to eradicate completely, however, policies based on outcomes are sensible alternatives. Not only are they objective and non-punitive, they may also create positive externalities that might not have been achievable using markets alone (see, e.g., Niederle, Segal, and Vesterlund 2013; Besley et al. 2017). For example, if journals clearly signal a determination to publish more female authors, they will likely decrease the relative price of co-authoring with women, encourage better co-author matches and increase the aggregate quality of economic research.
Nevertheless, an adequate understanding of the context in which discrimination occurs is absolutely crucial to responsible policymaking. More research is certainly needed. We hope journals are challenged to address the tougher standards they likely impose on women, willing to support the access and research needed to better understand them and open to whatever policy options most effectively check them.
Erin Hengel is a Lecturer in Economics at the University of Liverpool and has a PhD in Economics from the University of Cambridge. Eunyoung Moon is a Lecturer in Economics at the University of Liverpool and has a PhD in Economics from the University of Sussex.
References
Becker, Gary. 1957. The Economics of Discrimination. University of Chicago Press.
Besley, Timothy, Olle Folke, Torsten Persson, and Johanna Rickne. 2017. “Gender Quotas and the Crisis of the Mediocre Man: Theory and Evidence from Sweden.” American Economic Review 107 (8): 2204–42.
Blank, Rebecca M. 1991. “The Effects of Double-Blind Versus Single-Blind Reviewing: Experimental Evidence from the American Economic Review.” American Economic Review 81 (5): 1041–67.
Card, David, and Stefano DellaVigna. 2017. “What Do Editors Maximize? Evidence from Four Economics Journals.” Review of Economics and Statistics, no. forthcoming.
Card, David, Stefano DellaVigna, Patricia Funk, and Nagore Iriberri. 2020. “Are Referees and Editors in Economics Gender Neutral?” Quarterly Journal of Economics 135 (1): 269–327.
Ceci, Stephen J, Donna K Ginther, Shulamit Kahn, and Wendy M Williams. 2014. “Women in Academic Science: A Changing Landscape.” Psychological Science in the Public Interest 15 (3): 75–141.
Dion, Michelle L, Jane Lawrence Sumner, and Sara McLaughlin Mitchell. 2018. “Gendered Citation Patterns Across Political Science and Social Science Methodology Fields.” Political Analysis 26 (3): 312–27.
Ferber, Marianne A. 1986. “Citations: Are They an Objective Measure of Scholarly Merit?” Signs 11 (2): 381–89.
Gilbert, Julie R, Elaine S Williams, and George D Lundberg. 1994. “Is There Gender Bias in Jama’s Peer Review Process.” JAMA 272 (2): 139–42.
Hengel, Erin. 2019. “Publishing While Female. Are Women Held to Higher Standards? Evidence from Peer Review.” Mimeo.
Hengel, Erin, and Eunyoung Moon. 2019. “Gender and Quality at Top Economics Journals.” Mimeo.
King, Molly M, Carl T Bergstrom, Shelley J Correll, Jennifer Jacquet, and Jevin D West. 2017. “Men Set Their Own Cites High: Gender and Self-Citation Across Fields and over Time.” Socius 3: 1–23.
Lundberg, Shelly J. 1991. “The Enforcement of Equal Opportunity Laws Under Imperfect Information: Affirmative Action and Alternatives.” Quarterly Journal of Economics 106 (1): 309–26.
Lundberg, Shelly J, and Jenna Stearns. 2019. “Women in Economics: Stalled Progress.” Journal of Economic Perspectives 33 (1): 3–22.
Niederle, Muriel, Carmit Segal, and Lise Vesterlund. 2013. “How Costly Is Diversity? Affirmative Action in Light of Gender Differences in Competitiveness.” Management Science 59 (1): 1–16.
Sarsons, Heather. 2019. “Gender Differences in Recognition for Group Work.” Journal of Political Economy, no. forthcoming.
Tahamtan, Iman, Askar Safipour Afshar, and Khadijeh Ahamdzadeh. 2016. “Factors Affecting Number of Citations: A Comprehensive Review of the Literature.” Scientometrics 107 (3): 1195–1225.
The ProMarket blog is dedicated to discussing how competition tends to be subverted by special interests. The posts represent the opinions of their writers, not necessarily those of the University of Chicago, the Booth School of Business, or its faculty. For more information, please visit ProMarket Blog Policy.
